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Abstract 

We study lepton flavor violation (LFV) associated with tau leptons in the framework of the 
two Higgs doublet model, in which LFV couplings are introduced as a deviation from Model II 
Yukawa interaction. Parameters of the model are constrained from experimental results and also 
from requirements of theoretical consistencies such as vacuum stability and perturbative unitarity. 
Current data for rare tau decays provide substantial upper limits on the LFV Yukawa couplings in 
the large tan (5 region (tan (5 > 30), which are comparable with predictions in fundamental theories. 
Here tan (3 is the ratio of vacuum expectation values of the two Higgs doublets. We show that a 



I/-) ■ search for the LFV decays (p° — > t^u^ (r^e^) of neutral Higgs bosons ((/>° = h,H and A) at 

o ' 

I/"") ■ future collider experiments can be useful to further constrain the LFV couplings especially in the 



relatively small tan/? region (tan/? < 30), where rare tau decay data cannot give any strong limit. 
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I. INTRODUCTION 



Experimental determination of the electroweak symmetry breaking sector is important 
not only to confirm the Higgs mechanism and the mass generation mechanism for matter 
but also to obtain information for physics beyond the standard model (SM). Lots of new 
physics models predict extended Higgs sectors with more than one scalar doublets in the low 
energy effective theories. Such extended Higgs sectors would show distinctive features from 
the SM phenomenology The most obvious evidence is the confirmation of the existence of 
the extra scalar states such as CP-odd and charged states. Even when they are too heavy 
to be directly detected and only the lightest Higgs boson is found at experiments, we can 
explore a possibility of the extended Higgs sector by looking for deviations from the SM 
predictions in its couplings with gauge bosons and fermions as well as in the self coupling. 
Moreover, it can also be examined by searching for non-SM interactions. 

Lepton flavor violation (LFV) is an example for such non-SM phenomena. In particular, 
LFV in the Yukawa sector can only appear for extended Higgs sectors. Flavor violation 
between electrons and muonsQ] has been tested through rare muon decays such as fi — > c-f 
and u — > ee + e~, as well as through u-e conversion. Tau lepton associated LFV has also 
been studied by rare decays of tau leptons such as r — > iiP 
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and 



T 



a 



BBS, 



where 



1,2) respectively represent an electron 



and a muon, P° does 7r°, r) and 7/ mesons, M ± (M f± ) does tt^ and K ± mesons, and 
and fi ± . The LFV Yukawa couplings can be constrained from the data for these processes 
especially those with the Higgs boson mediation. For fi-e mixing, the Higgs boson mediated 



LFV coupling has been discussed in Ref. 
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ll|. Tau lepton associated LFV processes with 



the Hi ggs boson mediation have been discussed in models with supersymmetry (SUSY) 12, 
13, 14,|l5( as well as in the two Higgs doublet model (THDM) in some specific scenarios 1£, 
18]. In Ref. h|, tau associated LFV processes have been discussed comprehensively in 
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the framework of 4-Fermi contact interactions. Phenomenological consequences of the LFV 



Yukawa couplings associated with tau leptons have also been studied for future o 



13j,Q at (super) B factories [2l| and II 



Dserva 



igg s boson decays 
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Dies 
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271 and a muon collider 



such as B s decays 

CERN LHC2(|, an electron-positron linear collider (LC) 
addition, it has been pointed out that deep inelastic scattering processes fiN — > rX [29L 
from intense high energy muons at neutrino factories (or muon colliders) and eN — * tX 



25J at 
. In 
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by using the electron (positron) beam of a LC would be useful to further explore the tau 
lepton associated LFV Yukawa couplings. 

In this paper, we study LFV in Higgs boson decays into a t-Ii pair in the general frame- 
work of the THDM. In order to evaluate possible maximal values of the branching fractions, 
we first study experimental upper limits on the tau lepton associated LFV Yukawa couplings. 



The parameter space is tested by theoretical requirements for vacuum stability 



turbative unitarity 



LEP 3 



3J,|33|, 



311 ] and per- 



Current data from electroweak precision measurements at 



361 ] and those at the B factories 



37 



also strongly constrain parameters of the 
Higgs potential. Under these theoretical bounds and experimental limits on the model, pos- 
sible maximal values of the LFV couplings of are obtained by using the current data 
for rare tau decays, where <fr° represents two CP-even (h and H) and a CP-odd (A) Higgs 
bosons. We then evaluate branching ratios of <p° — > r^if with the maximal allowed values 
of the LFV couplings of r-£j-0° in a wide range of the parameter space. 

We here consider the model in which Higgs self-interactions and quark Yukawa inter- 
actions are invariant under the discrete symmetry ($i — > +$i and $2 — * —^2 with $ a 
(a = 1,2) being the Higgs doublets) in order to suppress flavor changing neutral current 
(FCNC)[39]. If the discrete symmetry is exact, only two choices are possible for Yukawa 



interaction; i.e., so called Model I and Model IIj40j. In Model I only one Higgs doublet 
gives masses of quarks and leptons, while one of the Higgs doublets gives masses of up-type 
quarks and the other does of down-type quarks and charged leptons in Model II. We assume 
the Model II Yukawa interaction for quarks assigning q l L — * +q l L , u l R — > —u R and d l R — > +d R . 
Even in Model II (or Model I), FCNC can be induced at one loop level due to off-diagonal 
elements of the Cabibbo-Kobayasi-Maskawa (CKM) matrix, and in case due to new physics 
effects. Flavor non- conserving decays of Higgs bosons into quark pairs have been studied in 



the THDM |16l. I41j] and also in the context of the SUSY models|42|. We do not discuss flavor 
violation in the quark sector in this paper. For leptonic Yukawa interactions, the discrete 
symmetry is assumed to be explicitly broken, so that the LFV Yukawa couplings naturally 
appear in the model. Once a fundamental model is specified at high energy scales, the LFV 
Yukawa couplings are predicted in terms of the model parameters. In the minimal super- 
symmetric standard model (MSSM) slepton mixing may be the origin of LFv[liQ,Q- 
Lepton flavor violating interactions are also induced at the loop level in the Zee model where 
tiny neutrino masses are explained by the dynamics of the extended Higgs sector 44J. We 
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here do not specify the origin of LFV, and treat the lepton flavor violating THDM as the 
low energy effective theory of such high energy theories. 

It turns out that rare tau decay searches can give substantial upper limits on the LFV 
couplings of T-£i-<f)° in the large tan (3 region, which are comparable with the values predicted 
by assuming some fundamental models beyond the THDM. Here tan/3 is the ratio of vac- 
uum expectation values for neutral components of the Higgs doublets. The upper limits are 
rapidly relaxed for smaller tan /3 values. Therefore, under the constraint from current exper- 
imental data and the theoretical bounds, sufficiently large branching ratios of 0° — > t if 
are possible except for extremely large tan/3 values, which can be tested at future collider 
experiments. We conclude that a search for these LFV Higgs boson decays can be useful to 
explore the LFV couplings especially in the parameter region where rare tau decay results 
cannot reach. In particular, the LFV decays of the lightest Higgs boson can be one of the 
important probes for extended Higgs sectors when the SM-like situation would be preferred 
by the data at the experiments. 

In Sec. |Hj the THDM with LFV in Yukawa interaction is defined. The allowed region 
of the parameters of the Higgs sector is discussed by requiring the theoretical consistencies 
and the experimental results. In Sec. Hill we discuss the upper limit on the LFV Yukawa 
couplings in a wide range of the parameter space by using the current data for the rare tau 
decays. In Sec. IIV1 LFV in Higgs boson decays is studied under the rare tau decay results. 
Conclusions are given in Sec. 

II. MODEL 

In this section, the lepton flavor violating THDM is introduced, and the theoretical 
constraints and the experimental limits are discussed. 

A. Lagrangian 

We consider the Yukawa interaction for charged leptons, 

-Aepton = lm {YeA^i + (^4 + 4 Y ^) $ *} ■ L J + Hc > W 

where $i and $2 are the scalar iso-doublets with hypercharge 1/2, (i=l-3) are singlet 
fields for right-handed charged leptons, Lj (i—1-3) denote the lepton doublets and are the 

4 



Yukawa couplings for i^. This interaction is reduced to be of Model II in the limit efj R — > 
with the discrete symmetry under e R — > +e R , Li — > +Li, $1 — > +$1, and $ 2 — > — $2- 
Nonzero values of efj R (i 7^ j) yield the LFV Yukawa couplings after the diagonalization of 
the mass matrix. We note that in supersymmetric standard models, the Yukawa interaction 



for leptons is of Mode 



II at the tree level, and e^ R can be induced at the loop level due 



nnn 

to slepton mixing|12j, HJ, |4j|. For the quark sector, Model II Yukawa interactions are 
assumed to suppress FCNC, imposing the invariance under the transformation of u l R — > —u R , 
d R -> +d R , q\ -> +qi, $1 -> +$1, and $ 2 -> -$ 2 - 

The Higgs sector of the general THDM is expressed as 

-£ffi gg s =m\ l^xl 2 + m\ |$ 2 | 2 - (m!$I$2 + H.c.) + y |$i| 4 + y |$ 2 | 4 
+ A 3 |$i| 2 |$ 2 | 2 + A 4 |$ t 1 $ 2 | 2 + |y ($I$ 2 ) 2 + H.c.j 
+ (A 6 |$!| 2 $}$ 2 + H.c.) + (A 7 |$ 2 | 2 $^2 + H.c.) . (2) 

In Eq. wij, A5, Ag and A7 are complex in general. We here assume that all the parameters 
m\_ 3 and Ai_7 are real. The terms of m§, A@ and A7 break the discrete symmetry explicitly. 
As we consider the model in which the discrete symmetry is explicitly broken only in the 
leptonic Yukawa interaction, we set the hard-breaking coupling constants to be zero in the 
Higgs potential; i.e., Ag = A7 = 1 , and retain only the soft-breaking mass parameter m\. 

There are eight degrees of freedom in the two Higgs doublet fields. Three of them are 
absorbed by the weak gauge bosons via the Higgs mechanism. Remaining five are physical 
states. After the diagonalization of the mass matrices, they correspond to two CP-even [h 
and H), a CP-odd (A), and a pair of charged (H ) Higgs bosons. The CP-even neutral states 
are defined such that h is lighter than H. In our model, the eight real parameters m1_ 3 and 
Ai_5 can be described by the same number of physical parameters; i.e., the vacuum expecta- 
tion value v (~ 246 GeV), the Higgs boson masses m h ,m H ,m A and m H ±, the mixing angle 
a between the CP-even Higgs bosons, the ratio tan/3 (= ($2) / i^i)) °f ^ ne vacuum expecta- 
tion values for two Higgs doublets, and the soft-breaking scale M (= \/ m\j sin (3 cos j3) for 
the discrete symmetry. The quartic couplings are expressed in terms of physical parameters 



1 Even in such a case, Xe and A7 arc effectively induced at the loop level. They are suppressed by the loop 
factor, so that we here neglect these small effects. 
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by|45] 



Ai 



1 



(— sin 2 f3M 2 + sin 2 am 2 + cos 2 am 2 ^) 



v 2 cos 2 (5 

= - ; 9 - (— cos 2 /5M 2 + cos 2 ttm^ + sin 2 arn^,) 



f 2 sin /? 



A, =^ 



-M 



sin 2a 



O Dill ^L-t / 

2m H ± H — : [m 



sin 2/3 



X A =— (M 2 +m 2 A -2m 2 H± ) , 

v 

\ 5 =\(M 2 -m 2 A ). 



(3) 
(4) 
(5) 

(6) 
(7) 



The relative size of the parameter M against v determines the decoupling property of 
the Higgs sector. For the case of M 2 3> v 2 , m H , m A and m H ± turn out to be around M. 
They decouple from low energy observables by a factor of v 2 /M 2 , and the lightest Higgs 



boson h becomes the SM-like one 



46|. The THDM effectively becomes the SM at low 



energies. On the contrary, when M 2 < v 2 , the Higgs boson masses can be varied by the 
variation of relative size of quartic couplings. In such a case, the parameter space is strongly 
constrained by the conditions from perturbative unitarity, which we shall discuss in the next 
subsection. Notice that even for M 2 < v 2 we can define the "SM-like" limit by setting 



sin(a — (3) ~ — 1 



45 



3- 



The coupling constants of hVV (VV = W + W~ and ZZ) then 



coincide with those in the SM at the tree level, and there are no HVV couplings. 



B. Constraint on the Higgs parameters 

Parameters of the Higgs sector are constrained from requirements of theoretical consis- 
tencies and also from the current experimental results. We here take into account two kinds 
of theorem, conditions; i.e., vacuu m staMitvQ and pertnrbative nnita r it y fl Q at 
the tree level. The condition for vacuum stability is expressed by 

Ax>0, A 2 >0, v / AiA^+A3 + MIN[0,A 4 + A 5 ,A 4 - A 5 ] > 0, (8) 

and that for tree-level unitarity is described as 

\{<PMA<Pi<t>2)\ <£, (9) 

where (0304|a°|0i</> 2 ) is the s-wave amplitude for the process of 0i0 2 0304 with a (a=l- 
4) denoting Higgs bosons and longitudinal components of weak gauge bosons. We employ 
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the conditions given in Refs. |33j, |45j in which the fourteen channels are taken into account 
in the THDM; i.e., ZZ, hZ, HZ, AZ, hh, HH, AA, hYLAiA, HA, W + W~, H+H~ and 
W ± H Zf . We take the criterion £ to be 1[32| (and also 1/2 40j for comparison). For the top 
and bottom Yukawa couplings with Higgs bosons (yu^o and y^a), we just put the criterion, 

|y ?#0 | 2 < 4tt, (q = t,b). (10) 

For example, this condition on y t f A and y h i A gives upper and lower limits for tan/3 (0.2 < 
tan/3 < 100-200) 2 . As shown in the next section, the conditions of Eqs. (JHJ) and © give 
more strict upper bound on tan (3 than that of Eq. (|1U|) in a wide parameter region. 

Next we consider the experimental constraints, which are provided by the LEP precision 
dataj^l, the b — > 57 results)^, and the direct search results for the Higgs bosons 3^|. 
The LEP precision data provide the strong constraint on the new physics structure via 
the gauge-boson two-point functions. The constraint on p parameter indicates that the 
Higgs sector is approximately custodial SU(2) symmetric. This requirement is satisfied 
when (i) m H± ~ m A , (ii) m H ± ~ m H with sin 2 (a — 0) ~ 1, and (iii) rn H± ~ m h with 
cos 2 (a — B) ~ l|36l. It is known that in Model II, the b — > 57 result gives the lower bound 

nn 

on the charged Higgs boson mass |3 71. 1381] . We here take into account this bound by requiring 
m H ± > 350 GeV. 



L Ft 

C. LFV parameters «;• •' 

The tau lepton associated LFV interactions in Eq. (JTJ can be reduced in the mass eigen- 
basis of each field to 

-£tLfv = ( K 3i TF Lii + k^P l t) {cos (a - (3) h + sin (a - (3) H - iA} 

V COS LJ 

\/2m 

" (^ i TP L V i + K^£ i P L V T )H- + R.C., (11) 



v cos 2 (3 

where Pl is the projection operator to the left-handed field, and t\ and £ 2 respectively 
represent e and /i. In general, the LFV parameters k^- R can be expressed in terms of efj R and 



2 Similar bounds are obtained for example in Ref. |4 
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tan j3: We here take these as effective couplings, and investigate their phenomenological 



consequences. We note that Eq. is exact in the limit of 



m 



0. The terms of nf 3 and 



v 3t 



1,2) are proportional to m e ., so that they decouple in this approximation. 



We briefly discuss relationship between k^- r and new physics models beyond the cut-off 
scale of the effective THDM. When a new physics model is specified at the high energy scale, 
nfj R can be predicted as a function of the model parameters. For example, in the MSSM, 
slepton mixing can be a source of LFV. Notice that the induced LFV Higgs interactions 
do not necessarily decouple in the limit where the SUSY particles are sufficiently heavy, 
because their couplings only depend on the ratio of the SUSY parameters. Therefore, the 
Higgs associated LFV processes can become important in a scenario with the soft-SUSY- 
breaking scale m SVSY to be much higher than the electroweak one. In the MSS M, p redicted 



values of \k^\ 2 can be as large as of C(10~ 6 ) when m SUSY is a few TeV I5I. |27| . In the 
MSSM with right-handed neutrinos, slepton mixing may be a consequence of running effects 
of the neutrino Yukawa couplings between the scale of the grand unification and that of the 



rr 



sleptons 



12 



13 



the Zee model 



OH 



right-handed neutrinos 43 ] . The parameters are mainly induced by mixing of left-handed 



The LFV Yukawa interactions can also appear effectively in 



The LFV parameters nfj R are induced through flavor violating couplings 
in the charged scalar interactions with leptons. 



III. BOUND ON LFV YUKAWA COUPLINGS FROM RARE TAU DECAYS 

In order to constrain the LFV parameters \k^\ and \k&\, we take into account the data 
for rare tau decay processes such as r — > £iP°, r — > £iM + M'~ , r — > £i£' + £'~ , and r — > ^7, 
where P° represents 7r°, 77 and 77' mesons, M^ 1 (M /=t ) does 7r =l= and K ± mesons, and l !± = 
e ± and u ± . The list of the current data from the B factories are summarized in Table HI 

These bounds may be improved at the super B factory around a 
digit (21 1. In our analysis, we take the underlined data in Table U] as our numerical inputs. 



J LFV parameters can be expressed as 

-I J 

{l + (43 + 4)*an/3} 
for the case of tan/3 < 0(1) which is satisfied in the MSSM case [H Itlj 



4 = —r ; r % ^2 ( X = L > R l (12) 
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Mode 




Belle (90% CL) 


BaBar (90% CL) 




— ^ p 


'7T° 


1.9 x l(r 7 [2] 




T 


— > 6 




2.4 x 1(T 7 [2] 




T 


— ^ p ~ 
7 o 


V 


10 x 10" 7 [2l 




T 






4.1 x 10- 7 [2l 




T 




'V 


1.5 x 10- 7 [2] 










4.7 x 10- 7 [2l 






— ^ 6 _ 


'7T + 7T~ 


8.4 x 10" 7 [3] 


1.2 x 10~ 7 [4J 


T 


^ 6 




5.7 x 10" 7 [3j 


3.2 x 10" 7 [4J 


T 


— ^ C 




5.6 x 10" 7 [3] 


1.7 x 10" 7 [4J 


T 


— ^ C 


K + K~ 


3.0 x 10" 7 [3j 


1.4 x 10" 7 [4J 


j- 






2.8 x 10- 7 [3l 


2.9 x 10" 7 [4] 


q-~ 






6.3 x 10" 7 [3] 


2.6 x 10" 7 [4J 


q-~ 


— > 11 




15.5 x 10" 7 [3] 


3.2 x 10" 7 [4J 


T 


— > [1 




11.7 x 10~ 7 [3] 


2.5 x 10 _7 [4J 


T~ 


— > 6~ 


'e + e~ 


3.5 x 10" 7 [5] 


2.0 x 10- 7 [6] 


T~ 


— > e~ 




2.0 x 10 _7 [5l 


3.3 x 10 _7 [6] 


r~ 


-> M" 


~e + e~ 


1.9 x 10- 7 [5] 


2.7 x 10" 7 [6J 


T~ 


-> 




2.0 x 10" 7 [5] 


1.9 x 10" 7 [6J 


T ~ 


-> e7 




3.9 x 10- 7 [Z1 




T - 


i"7 




3.1 x 10~ 7 [8J 


6.8 x 10" 8 [9j 



r->/i7 3.1 x 10 - ' I8J 6.8 x lQ-° [flJ 

TABLE I: Current experimental limits on branching ratios of the LFV rare tau decays. 



The branching ratios for these r decays are calculated in our model as 

B / - _^ n— o\ = 3G 2 F m 3 r ml F%T T \n 3i \ 2 f _ 2 ( m u cot g - m d tan /? \ 2 1 

nT ^ j7Fj 32ttcos 4 /? V m r ) V m u + m d j raV 1 J 

Br(r~ -> e~T]) ~ ^ G F m T m i F v T ' r l^ 3 ^ A m v\ * ( m u cot /3 + (m d - 2m s ) tan(3\ 2 1 



32n cos 4 /3 V m 2 . ) \ m u + m d + 4m s / ' 

(14) 

_. , ._ , _v G \m\B 2 r T \n3i\ 2 [ /sin a cos(a — 8) cosasin(a — 8)\ 

Br(r -> ^ tt+tt ) = — — — — <^ m d 



2567T 3 cos 6 /5 i " V m h 



9 



Br I 



T 



Br(r" 
Br(V 



— m u cot j3 
v-K+K-) - 

— m u cot (3 

9-0'+ B'-\ 



cos a cos a 



nir 



( .. 

_ G 2 F m 5 T B 2 r T |/c 3 ,| 2 
256tt 3 cos 6 

/ cos a cos(a — (3) 
V rn 2 h 

Bt(t- -> riiT+Tr" 



/3) sin a sin(a — /3) 



m 



H 



(15) 



sin a cos(a — (3) cosasin(a — (3) 



mi 



m 



H 



sin a sin (a 



P) 



m 



n 



G%m 



32tt 4 



G 2 F mj,mlr T \K 3 i\ 2 j^sin a cos(a — (3) cos a sin(a — /3) ^ 2 | sin 2 /3 
1536tt 3 cos 6 /3 



Br(r~ 



G 2 F m 2 e W T T T \n 3i \ 
2048vr 3 cos 6 /? 



sma cos a 



+ 



(16) 
(17) 

(18) 



P) 



cos a sin a 



{a-f3)\ 2 sin 2 /3 



+ 



2 /sin a cos(a — (3) cosasin(a — /3)\ sin/3 



mt 



m 



n 



(19) 



Bt(t- 



a em G\m\r T 
72(4tt) 4 cos 6 /? 



3; 



sin a cos a 



(3) r . , . cos a sin(a 
— /_(x h ) 



0) 



m 



f-(x H ) 



sin /3 



/VOa) 



I R ,2 /sin a cos(a — /3) . . cos a sin(a — (3) sin/3 sin/3 

+ K 3 | ( J~\ X h) Zl2 J-{ X h) + —^2~J+{Xa) ~\ o 



mi 



m 



H 



m 



m 



(20) 



where |ft 3 i| 2 = | | 2 + |ft§| 2 , Gf is the Fermi constant, F n and F n are the decay constants 
of the pion and the r] meson which are defined as shown in Ref . [19I] . t t is the life time 
of the tau lepton, and a cm is the fine structure constant. The expression in Eqs. (JUjl 
is deduced in the approximation of non- mixing between the octet and singlet states; i.e., 
rj = f]s- In Eqs. (|15|) and (|16j) . B is the matrix element for a pair production of pions, 
which is evaluated at the leading order as B = m 2 ±/ (m u + m^) (= m 2 K± / (m u + m s )) by 



using the chiral LagrangianQ]. In Eq. ()18j) . I' represents either e or /1 but £' 7^ £{. In 
Eq. pnjl . fJx) = 1 ±3(3 + 2 lna) and = m 2 T /m 2 r (</>° = h, H and A) 4 . We here omit the 
sub-leading contributions such as 0(m e Jm T ) 5 . We also do not consider the contribution to 



4 In this calculation, masses of neutrinos are ignored. 

5 The crossing terms between and nf^ do not appear in this approximation. 
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t — > £{P®, t — > ii£' + £'~ and r — > £iM + M'~ from gauge boson mediated LFV diagrams which 
are induced at the loop level. In the SUSY-like limit (m# ~ m A ), Eq. (fT9"j) coincides with the 
results of Ref. Q| 6 , and the ratio between Eq.(|14j) and E g. (1191) reproduces the result shown 
in Ref. [l4|. Our result of Eq. JE) is consistent with Ref. |lj|, and Eqs. (fTTTl). (fHJ) . <fTT>l> - and 
(jlb|) are also consistent with the results in Ref. [3] up to the colour factor 7 . We keep the 
terms with O(mp /ffiJ) in Eqs. (|13|) and ()14|) although we ignore those with Q{m\. f lrni) in 
the derivation of Eqs. (fTB]) and (fTB|) . The expression of Eq. (|2Uj) is compatible with Ref. jl8[. 
The branching ratios of the r + decay are the same as those of r~ in Eqs. fll3|) - pnj) . 

Some comments are in order. First, the processes r — > £iP° (P = tt°, tj and rj') are 
mediated only by the CP-odd Higgs boson A, and their branching ratios strongly depend on 
masses of the outgoing mesons. Because of the cancellation between the singlet and octet 
component in the r/ meson, the bound on \k^\ 2 from r — > lit]' may be less important than 



that from r — > £iTj 15[. We do not include r — > 1$)' in our analysis below. From Tableland 
Eqs. fl~3|) - fT4"|) . it is expected that r — > £iT\ gives the most stringent upper limit on |K 3 j| 2 of 
these three modes. Second, the expressions for the branching ratios of r — > £iM + M~ with 
M ± = 7r ± and are the same up to the difference of down type quarks d and s as seen in 
Eqs. ()15|) and ()16|) . These modes are mediated by the CP-even Higgs bosons h and H. As the 
mass of the s quark is much greater than that of the d quark while the experimental upper 
limits on them are the same order, r — > £iK + K~ gives the more stringent bounds on \k^\ 2 
than r — > £iH + n~ in a wide region of the parameter space. It is also found that r — > £ i 7r ± K Zf 
cannot contribute to limit values of |k3j| 2 . Their branching ratios are one- loop induced and 
much smaller than those of r — > £^K + K~ due to the loop suppression factor and the CKM 
factor iKidKTJ 2 ; while the data for these processes are the same order: see Eq. (|T7j) . Third, 
in Eqs. (|TH|) and ()19|) the branching ratios of r~ — > £^£' + £'~ are proportional to the mass 
of £'. As the data for r~ — * and r~ — > £~e + e~ are the same order, r~ — > £^fi + fi~ 

provide much stronger upper limits on \K3i\ 2 within the tree approximation because of the 
mass difference between electrons and muons 8 . The modes r~ — > £^£' + l'~ are mediated by 
all neutral Higgs bosons h, H and A. Finally, r~ — > are one-loop induced processes in 



6 Eq. (|19f) is also consistent with the expression in Tief \p c k 

7 The color factor is missing in the expressions in Ref. [19j. 

8 At the loop level, the branching ratio of the r~ — > £~e + e~ process can become lar ger than that of 
t~ — > £r because of the phase space dependence in the photonic penguin diagram|23j. 
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our model. All kinds of the Higgs bosons are included in the one-loop diagrams. Notice that 
as shown in Eq. ()20j) the one loop diagram with a charged Higgs boson is asymmetric with 
respect to chirality of the tau lepton, as the neutrino in the diagram is left-handed. In the 
following, we study the bound on | | 2 , and do not treat \^\ 2 and |fi^|| 2 separately. 

Since branching ratios for r~ — > £^P°, t~ — > £~ M + M~ , t~ — > £~£' + £'~ and t~ — > £~^ 
depend on different combinations of the Higgs boson masses, independent information can be 
obtained for the model parameters by measuring each of them. When all the masses of Higgs 
bosons are large, these decay processes decouple by a factor of l/^Higgs- Although these 
branching ratios are complicated functions of the mixing angles, each of them can be simply 
expressed to be proportional to tan 6 (3 for tan/3 3> 1 in the SM-like region (sin(a — (3) ~ — 1). 
This tan 6 (3 dependence is a common feature of the tau-associated LFV processes with the 
Higgs-mediated 4- Fermi interactions 9 . 

The experimental upper limit on | | 2 can be obtained by using the experimental results 
given in Table H] and analytic expressions for the decay branching ratios in Eqs. fT3|) - (|20|) . 
For instance, let us consider the bound from the r — > fir] results jlij]. From Eq. ((Hj) we 
obtain 

I ,2W, max,2x 1 287rBr (r -» fxr]) cxp m\ cos 6 (3 

«32 < (J K 32 I )r^un = 7 2X 2 • 2 a » I 21 ) 

m 9Glml m ^r T (l - f ) sin ? 

where Br(r — > fir]) cxp is the experimental upper limit on the branching ratio of r — > fir) in 
Table |U In particular, for tan (3 3> 1, the right-hand-side can be expressed by 

* -^Maniy 1 ^) 6 - (22) 



I max 
l ft 32 



T->fir) 

_max 1 2 



It can be easily seen that the bound (|^32 ax | ) T -> m is rapidly relaxed in the region with small 
tan/3 and large m A . In a similar way to Eq. (|21|). the maximal allowed value (l^^l 2 )^^ 
can be calculated for each mode. The combined upper limit (|ft^ ax | 2 ) is then given by 

i4Tf = Mm{(Kr\ 2 ) T ^A\^r\ 2 ) T ^ + ,- , (Kr\ 2 ) T ^ K+K - , (i«ri 2 w- • ■} • 

(23) 



9 This also applies to the one- loop induced r — ► processes whose diagrams have the same tan (3 depen- 
dence. 
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FIG. 1: Contours of |K^ ax | 2 , the possible maximal value of | AC32 1 2 from the rare tau decay results, 
are shown (a) in the tan P-tua plane and (b) in the tan /3-sin 2 (a — (3) plane. The parameters are 
taken to be (a) m h = 120 GeV, m H = m H ± = 350 GeV and sin(a — (5) = —0.9999, and (b) 
m h = 120 GeV and m H = tua = m H ± = 350 GeV. The remaining parameter M is scanned from 
to 1,000 GeV. The dark (light) shaded area indicates the excluded region by the theoretical 
requirements of Eqs. ©, © and (JTUJ) with f = 1 (£ = 1/2). 

As shown below, r — * if) and r — > iff give the strongest upper limits on |K 3i | 2 in a wide 
range of the parameter space. In addition, in some parameter regions r — > i i K + K~ and 
t — > i i p + fi~ can also give similar limits on \n3i\ 2 to those from the above two processes. 

In Figs, d-(a) and[TJ-(b), contour plots for | shown under the rare tau decay 

results in the tan^-m^ plane and the tan/5-sin 2 (a — j3) plane, respectively. The combined 
excluded region from the theoretical conditions in Eqs. (jH)), Q and ()10j) is indicated by the 
dark shaded area for the criterion £ = 1 and by the light one for £ = 1/2. In Fig. E-(a), 
parameters of the Higgs sector are taken to be m h = 120 GeV, m H = m H± = 350 GeV and 
sin(a — (3) — —0.9999. In Fig. IU-(b), those are m h = 120 GeV and m H = m A = m H± = 350 
GeV. The value of |^32 ax | 2 is independent of M, the soft-breaking scale of the discrete 
symmetry. On the other hand, theoretical bounds from vacuum stability and perturbative 
unitarity are sensitive to M. Therefore, we evaluate such a theoretical allowed region by 
scanning M to be from to 1000 GeV. We also take into account the constraint from the p 
parameter measurement and the b —* S7 result by taking sin (a — f3) ~ —1 and m H = m H ± 
with m H ± > 350 GeV for Figd-(a), an d m A = m H± with m H± > 350 GeV for Fig[T]-(b). 
From the both figures, it is easily found that the value of |ft!£> ax | 2 can extensively be larger for 
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m A [GeV] sin 2 (a-^) 




FIG. 2: Contour plots of | 

^32 I £IX^ 

shown (a) in the tan 0-rriA plane and (b) in the tan /3-sin 2 (a— f3) 
plane similar to Figs. and^-(b) but with different parameter choices; (a) m h = 120 GeV, 

m H = mjj± = 600 GeV and sin(a — ft) = —0.9999, and (b) m h = 120 GeV and m H = tua = 
itih± = 600 GeV. The dark (light) shaded area indicates the excluded region by the theoretical 
requirements of Eqs. @, © and (JTHJ) with £ = 1 = 1/2). 

smaller tan/? in the allowed region under the theoretical constraints. For tan/5 < 10 (30), 
l K 32 ax | 2 can be O(0.1) (O(10~ 4 )). Among the rare tau decay processes, r — > and r — > fi^y 
provide the most stringent constraints on |ft 32 | 2 - While r — > \ir\ is mediated only by A, 
r — > /i7 depends on the masses of fo, iJ, A and if ± . For sin 2 (a — /3) ~ 1 and ~ m^, the 
branching ratio of r — > /i7 in Eq. PU|) is suppressed because of the cancellation between the 
one-loop diagrams of A and H . Therefore, |k 32 | 2 is bounded most strongly by the r — > firj 
result for this case 10 . When ttia differs from ran or when sin 2 (a — /?) is to some extent 
smaller than unity, the one-loop induced r — > fi'y process becomes important, and gives the 
most stringent bound on |«; 32 | 2 of all the tau rare decay processes. 

In Fig. El-(a) and|2]-(b), the similar contour plots are shown with assuming larger values 
of ra H and ra H± ; i.e., ra H = ran± = 600 GeV with sin(a — (3) = —0.9999 for Fig. EK a ) and 
ran = m A = m H ± = 600 GeV for Fig. 0- (b), respectively. The magnitude of l^l^f*! 2 and 
its tan/3 dependence are similar to the case of Fig. Q-(a) and^-(b), although the theoretical 
allowed regions become changed to the considerable extent. 

In Figs. ^ and 121 the value of | ^§2 ax | 2 can De much larger than 100 in a wide range of the 
parameter region. One might think that such large values of |K 3 j| cannot be consistent with 

10 The MSSM result approximately corresponds to this casefl^. 
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the unitarity argument for the LFV Yukawa couplings. However, it should be emphasized 
that the above figures show the contour plots for |ft^> ax | 2 under the rare tau decay results, 
and not for l^] 2 - The region of |^32 ax | 2 > 1 should be taken as the area where | K32 1 2 can be 
as large as (9(10~ 2 -10~ 4 ) easily. It is concluded that current results of the tau LFV decays 
do not give any substantial upper limit on |k 32 | 2 except for high tan/3 region (tan/3 > 30). 

Finally, we comment on the bound on |ft3i| 2 , the LFV parameters for r-e mixing. Similar 
to r-fi mixing, we can discuss |^|^ ax | 2 comparing the data of r — > er], r — > efi + fi~, r — * 
eK + K~ and r — > listed in Table H] with the formulas given in Eqs. ([14 )1 - (|2U |) . These 
formulas for r-e mixing are common with r-/x mixing except for the factor of |«3i| 2 , so that 
difference in contours of l^^l 2 from those of |^3^ ax | 2 only comes from that in the data. 
In Table [Q the experimental limit for the branching ratio of r — > er] is about 1.5 times 
weaker than that of r — > fir], while that of r — > e^ is 5.7 times relaxed as compared to that 
of r — > fi'y. Moreover, the upper limit on Br(r~ — > e~K + K~) is 1.8 times stronger than 
that on Br(Y~ — > ]i~K + K~). We have numerically confirmed that there are some regions 
where r~ — > e~K + K~ can give the most stringent bound on |ft3i| 2 . Therefore, | Ac§^ a;x | 2 is 
determined from one of r — > er], r — > e^ and r~ — > e~ K + K~ depending on parameter 
regions. 



IV. LEPTON FLAVOR VIOLATING HIGGS BOSON DECAYS 



As shown in the previous section, the LFV Yukawa couplings can be tested only in the 
large tan (3 region by searching for rare tau decays. In order to cover the region unconstrained 
by rare tau decay results, we here consider LFV via the decay of the neutral Higgs bosons; 



i.e., 



r^if (0° = h,H and A). Branching ratios for these decays are calculated 



3, 
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\M to be 
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1 m 2 cos 2 (a-P). | 2 
167T v 2 cos 4 P 
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FIG. 3: Contour plots of Br(/i — ► r =t /i :F ) max , the possible maximal values for the branching ratio 
under the tau rare decay results, are shown (a) in the tan/3-m^ plane and (b) in the tan/3-sin(a— (3) 
plane. The parameters are taken as the same as Figs. [D (a) and^Qo), respectively. The dark (light) 
shaded area indicates the excluded region by the theoretical requirements of Eqs. (JSJ), @ and (fTU|) 
with £ = 1 (£ = 1/2). 

where r(0° — > all) is the total width for corresponding neutral Higgs boson 0°. We here 
neglect terms of O(m|./m| )- Branching ratios for 0° — > t + £~ coincide with those for 
<p° — > r~£f given in Eqs. (J2U), (j2I3) an d (EHI)- In the following, we concentrate on the 
decays into a r-/i pair. We take the values of the SM parameters as a cm = 0.007297, 
G F = 1.166 x 10~ 5 GeV" 2 , m z = 91.19 GeV, m T = 1.777 GeV, m M = 0.1057 GeV, m b = 4.1 
GeV, m t = 174.3 GeV, and m c = 1.15 GeV. 

A. The LFV decay of the lightest Higgs boson 

A search for the LFV decays h — > t if can give important information for extended 
Higgs sectors and thus for the structure of new physics, even when only h is found and 
any other direct signals for the extended Higgs sector are not obtained by experiments. We 
here evaluate the possible maximal value of the branching ratio Br(/i — > r~yU + ) max under 
the results of the rare tau decay search, by inserting |K^ ax | 2 of Eq. (}23|) into the I k^q 1 2 in 
Eq. (J24)) . In calculation of T(h — * all), the decay modes of bb, cc, t + t~, 77, gg, Z^f[A0} and 
T^tf are taken into account. 

In Figs. Ol-(a) and HD-(b), contours of Br(/i — > r ± /i :F ) max , which is twice of Br(/i — * 
r~/i + ) max , are shown in the tan /3-m A plane and in the tan/3-sin 2 (a — (3) plane, respectively. 
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The parameters are taken to be the same as those for Figs.[T]-(a) and[T]-(b), respectively; i.e., 
(a) m h = 120 GeV, m H = m H± = 350 GeV and sin(a - (3) = -0.9999, and (b) m h = 120 
GeV and m H = m A = m H± = 350 GeV, with M to be scanned from to 1000 GeV. We 
again show the excluded area from requirements of tree-level unitarity and vacuum sta- 
bility as in the same way as Figs. [TJ-(a) andd-(b). For low and moderate values of tan/3 
(tan/3 < 30), where rare tau decay results cannot give substantial upper limit on |«3 2 | 2 , 
Bi(h — > r ± /i T ) max can be sufficiently large. We find that the possible maximal values of the 
branching ratio can be greater than O(10~ 3 ) in a wide rage of the allowed region under the 
theoretical conditions in Eqs. (jHJ), © and fTUJ) . 

In Fig. IU-(a) and0]-(b), similar contour plots of Br(/i — > r ± /i =F ) max are shown for the 
same parameters as Figs. 121(a) and|21-(b), respectively; i.e., (a) m H = m H ± = 600 GeV with 
sin(a — (3) = —0.9999, and (b) rrin = rn A = mu± = 600 GeV, with M to be scanned from 
to 1000 GeV. The excluded area from requirements of tree-level unitarity and vacuum 
stability are also shown. The possible maximal values for the branching ratio are similar to 
those in the case shown in Figs. Ol-(a) andE3-(b). In the allowed region under the conditions 
in Eqs. ©, © and (HH, Br(/i -> r ± // =F ) max can be greater than O(10" 2 ). 

For relatively lower tan/5 values, the experimental upper limits on | | 2 from rare tau 
decays are weaker, and Br(h — * T ± /i :f ) can be sufficiently large (> (9(10~ 3 ) for rrih ~ 120 
GeV). It is expected that a sufficient number of such light h can be produced at future 
colliders such as CERN LHC, currently planned International Linear Collider (ILC) and 
CERN CLIC. It has been pointed out that the decay process h — > r ± /i :F can easily be 
detected at ILC with the luminosity of 1 ab" 1 , when rrih ~ 120 GeV and Br(/i — > t pF) ^ 
O(10~ 3 ) n via the Higgsstrahlung process by using the recoil momentum of Z boson 27 1. 
Therefore, the LFV search via the decay h — > r ± /i T at ILC can be complementary to that 
via rare tau decays at (super) B factories, and the both cover a wide region of the parameter 
space of the lepton flavor violating THDM. 



11 Below that value, the signal would be seriously suffered from the backgrounds. In particular, events from 
h — > and h — > t + t~ would be difficult to be separated from the signal. The branching ratio of 

h — * is approximately proportional to m^j(N c m^) for a relatively light h, with N c being the color 

factor; i.e., Br(h — > ~ 3 x 10~ 4 . For a large tan/3 value with sin 2 (a — (3) <C 1, the maximal value 

of Br(/i — > t^/m^) becomes smaller and can be comparable to Br(/i — > /x + /i~). A simulation study is 
necessary to clarify this point. 
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FIG. 4: Contour plots of Br(ft, — ► T =t /i :F ) max similar to Figs.OKa) and|31-(b) are shown (a) in the 
tan (3-m a plane and (b) in the tan/3-sin 2 (a — (3) plane with the parameters taken as the same as 
Figs. GK a ) and 121(b), respectively. The dark (light) shaded area indicates the excluded region by 
the theoretical requirements of Eqs. ©, © and lfTU|) with £ = 1 (£ = 1/2). 

B. The LFV decay of Heavier Higgs bosons 



Next we discuss branching ratios for the LFV decays of heavier Higgs bosons, H/A — > 
T^if, using Eqs. and under the current data of LFV rare tau decays. In the 
THDM, there are many possible decay modes for H depending on the mass spectrum; i.e., 
hh, AA, hA, hi, hZ, A*y, AZ, H+H~ and H ± W T as well as // (/ = t,b,c,r), W^W*, 
ZZ, Zj, 77 and gg. The last three modes as well as hA, h^, A^ and hZ appear through 
the one-loop induced couplings 12 . Those for A are hZ, HZ, H + H~ , H ± W T and // at the 
tree level as well as hh, hH, HH, hj, Hj, W ± W T , ZZ, Zj, 77 and gg at the one loop 
level 4(|- 

The branching ratios for H/A — > t if are sensitive to the masses of all the Higgs bosons. 
Here we consider the case of sin(a — (3) = —1 and m H = m A = m H± (= m$). As discussed in 
Sec. II, the p parameter constraint is satisfied for this choice. From the b — > 57 results, m $ is 
taken to be greater than 350 GeV. As also discussed in Sec. II, M determines the decoupling 
property of heavier Higgs bosons. Although the branching ratios Bt(H/A — > r^tf) are 
insensitive to M in the present parameter set, its value strongly affects the allowed parameter 



12 In the numerical analysis, we included contributions from these Zy, 77 and gg modes in addition to all 
the tree level modes, but neglected the other loop-induced modes. 
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FIG. 5: Contour plots of Br(if — > r /x :f ) max are shown in the tan/3-m$ plane (m $ = m H = 
m A = m H ±) for m h = 120 GeV and sin(a — j3) = — 1 with (a) M = m$, (b) M = m^/y^, and (c) 
M = 0. The dark (light) shaded area indicates the excluded region by the theoretical requirements 
of Eqs. ©, © and flE) with £ = 1 (£ = 1/2). 

region under the theoretical conditions of Eqs. (JHJ) and 0. Notice that couplings of if are 
similar to those of A for sin(a — f3) = — 1 where there are no HVV couplings. Hence we 
show the results only for the LFV decays of H below. In a general case, the branching ratio 
of H — > T^fi 1 " tends to be smaller than that of A — > due to the contribution from the 

modes H — ► VV. 

In Figs. El-(a), E3-(b) andEJ-(c), contour plots of Br(H — > r ± /i :F ) max , the upper limit of 
Br(H —> r^fi 11 ) under the rare tau decay results, are shown in the tan/3-m$ plane for 
M — ttiq, and 0, respectively. As expected, the contours are insensitive to the 

values of M, and approximately the same in Figs. 03-(a), EJ-(b) and^Kc). It is shown that 
Bi(H — >• r ± / u =F ) max can be larger than 10~ 3 except for large tan/5 values with relatively 
small m $ . Therefore, it turns out to be no substantial upper limit on the By(H — > r ± /i =F ) 
in the relatively low tan (3 region (tan (3 < 20) from the LFV rare tau decay results. When 
M is smaller than m$, where the heavier Higgs boson partially receive their masses from 
the vacuum expectation value, the allowed parameter region is strongly constrained by the 
requirements of vacuum stability and perturbative unitarity. In particular, for M = 
(Fig. EJ-(c)), the allowed region is limited only the area of around tan/3 ~ 1 and m$ < 600 
GeV. 
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The extra Higgs bosons (H, A and H ) are expected to be searched at the LHC The 
signal of gg — > iJA4 — > r ± /i =F may be detectable at LHC with high luminosity (100 fb _1 ) 
when Bt(H/A — > r ± /i =F ) is greater than 10~ 2 for rn H ^ A ~ 350 GeV and tan/3 = 45 
However the rate is rapidly reduced for smaller values of tan (3 and for larger values of m H / A - 
Further feasibility study is necessary. 



V. CONCLUSIONS 



Lepton flavor violating decays of Higgs bosons have been studied in the framework of 
the THDM, in which LFV couplings are introduced as a deviation from Model II Yukawa 
interaction in the lepton sector. The model parameters are constrained by requirements 
of tree-level unitarity and vacuum stability, and also from the experimental results. The 
parameters | ^t 3i | 2 in LFV Yukawa interactions are bounded from above by using the current 
data for rare tau LFV decays. Each process is mediated by the different combination of the 
Higgs bosons, so that the data for each of them provides independent information to the 
lepton flavor violating THDM. It has been found that among the rare tau decay data those 
for t — > If] and r — > Ifj give the most stringent upper limits on |K 3 j| 2 in a wide range of 
the parameter space. 

In the large tan/3 region (tan/3 > 30), the upper limit on \k^\ 2 due to the rare tau decay 
data turns out to be substantial and comparable with the value predicted by assuming some 
fundamental theories such as SUSY. The upper limit would be improved in future by about 
one order of magnitude at the experiment at (super) B factories. For smaller values of tan j3, 
the upper limit is rapidly relaxed, and no more substantial constraint is obtained from the 
rare tau decay results. 

We have shown that a search for the LFV decays 0° — > t if of neutral Higgs bosons 
(0° = h,H and A) can be useful to further constrain the LFV Yukawa couplings at future 
collider experiments. In particular, the LFV decays of the lightest Higgs boson can be 
one of the important probes to find the evidence for the extended Higgs sector when the 
SM-like situation would be preferred by the data at forthcoming collider experiments. The 
branching ratio for h — > r ± /i T can be larger than O(10~ 3 ) except for the high tan/3 region 
under the constraints from the current experimental data and also from the theoretical 
requirements. At ILC (and in case at LHC), such a size of the branching fractions can be 
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tested. Therefore, we conclude that the search of LFV in the Higgs boson decay at future 
colliders can further constrain the LFV Yukawa couplings especially in the relatively small 
tan/3 region (tan/3 < 30), where rare tau decay data cannot reach. 
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